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(1) Real Party in Interest 

The real party in interest is Microsoft Corporation, the assignee of all right, 
title and interest in and to the subject invention. 

(2) Related Appeals and Interferences 

Appellant is not aware of any other appeals or interferences which will 
directly affect, be directly affected by, or otherwise have a bearing on the Board's 
decision to this pending appeal. 

(3) Status of Claims 

Claims 1-5, 13-25, 27-33, and 35-42 stand rejected and are pending in this 
Application. Claims 1-5, 13-25, 27-33, and 35-42 are appealed. Some of claims 
1-5, 13-25, 27-33, and 35-42 were previously amended. Claims 6-12, 26, and 34 
were previously canceled. Claims 1-5, 13-25, 27-33, and 35-42 are set forth in the 
Appendix of Appealed Claims on page 26. 

Claims 1 and 3-5 stand rejected under 35 U.S.C. § 103(a) as being 
unpatentable over U.S. Patent No. 5,838,3 13 to Hou et al. (hereinafter "Hou") in 
view of U.S. Patent No. 6,081,829 to Sidana (hereinafter "Sidana"). 

Claims 2, 17-25, 27-33, and 35-42 stand rejected under 35 U.S.C. § 103(a) 
as being unpatentable over Hou in view of Sidana in further view of U.S. Patent 
No. 6,009,462 to Birrell et al. (hereinafter "Birrell"). 

Claims 13-16 stand rejected under 35 U.S.C. § 103(a) as being unpatentable 
over Hou in view of Sidana in view of Birrell in further view of U.S. Patent No. 
5,526,407 to Russell et al. (hereinafter "Russell"). 
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(4) Status of Amendments 

A Final Office Action was issued on June 3, 2003. 

Appellant filed a Notice of Appeal on December 3, 2003 in response to the 
Final Office Action. 

No amendments have been filed subsequent to the Final Office Action of 
June 3, 2003. 

(5) Summary of Invention 

A network system includes a client computer (15) that can present 
multimedia content to a user and an annotation server (10) that can transmit, to the 
client computer (15), annotations for the multimedia content. Users are presented 
with annotations generated by other users and can create their own annotations 
that can be presented to others. Users can be notified of new annotations by 
electronic mail (email) (352, 354), and can also create new annotations by email. 

According to some aspects of the invention, a user creating a new 
annotation can identify one or more other users that are to be notified (266), by 
email (352, 354), of the new annotation. Users to be notified by email can be 
identified manually by the annotation author, or alternatively a default user(s) to 
be notified by email can be associated with the media content. Additionally, group 
identifiers can be used to identify groups of users to be notified of the new 
annotation by email. 

According to other aspects of the invention, an email message (302, 332) 
notifying a recipient of a new annotation also includes a user-selectable link for 
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the temporal segment of multimedia content corresponding to the annotation. 
Upon receipt of the email message, the recipient can select the link (306, 358) and 
have the media server (11) present, to the user, the temporal segment of 
multimedia content that corresponds to the annotation. 

According to other aspects of the invention, a new email message can be 
generated that includes the content for a new annotation (276) but that is not in 
response to an email notification of a new annotation. This new email message 
includes the content for the new annotation as well as information that can be used 
by the annotation server to create a new annotation (404, 406). Examples of such 
information include the multimedia content to which the new annotation 
corresponds, and the temporal range of the media content to which the new 
annotation corresponds. 

According to other aspects of the invention, one or more annotations can be 
created by forwarding an email thread (374, 378, 382) to the annotation server 
(10). Upon receiving an email thread, including multiple email messages that 
were forwarded or replied to by various users, the annotation server (10) creates a 
new annotation including all of these email messages (406). Alternatively, a 
separate annotation for each email message in the thread may be created. 

According to other aspects of the invention, an email message (302) can 
include the content of a new annotation (304) as well as an identifier (306) of the 
media content associated with the new annotation and an installation option (314). 
The installation option identifies an installation program that can be used to install 
one or more modules allowing the content and corresponding media content to be 
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rendered. The installation option can take a variety of different forms, such as a 
user-selectable link that identifies the installation program. 

(6) Issues 

1 . Whether claims 1 and 3-5 are unpatentable over Hou in view of 
Sidana when the combination of references does not disclose or suggest the aspect 
of an annotation server to analyze electronic mail messages that are received not in 
reply to a previous electronic mail message that included annotation data, to locate 
data in the electronic mail messages, and to generate new annotations that include 
the located data but no new annotation data. 

2. Whether claims 2, 37-39, and 41-42 are unpatentable over Hou in 
view of Sidana in further view of Birrell when the combination of references does 
not disclose or suggest the aspect of an annotation server to analyze electronic 
mail messages that are received not in reply to a previous electronic mail message 
that included annotation data, to locate data in the electronic mail messages, and to 
generate new annotations that include the located data but no new annotation data. 

3. Whether claims 17-25 and 27-30 are unpatentable over Hou in view 
of Sidana in further view of Birrell when the combination of references does not 
disclose or suggest the aspect of media content, being accessed for rendering to a 
user, beginning with the one of a plurality of temporal segments of the media 
content that corresponds to a new annotation. 

4. Whether claims 3 1 and 32 are unpatentable over Hou in view of 
Sidana in further view of Birrell when the combination of references does not 
disclose or suggest the aspect of displaying a default recipient, corresponding to 
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the identified one or more annotation sets, that is to receive an electronic mail 
notification of the new annotation. 

5. Whether claims 33 and 35-36 are unpatentable over Hou in view of 
Sidana in further view of Birrell when the combination of references does not ' 
disclose or suggest the aspect of creating, after the plurality of electronic mail 
messages have been included in the electronic mail thread, a plurality of 
annotations from the plurality of electronic mail messages in the electronic mail 
thread. 

6. Whether claims 13-16, and 40 are unpatentable over Hou in view of 
Sidana in view of Birrell in further view of Russell when the combination of 
references does not disclose or suggest the aspect of including an installation 
option in an electronic mail message that identifies an installation program that 
can be used to install one or more modules allowing the content of a new 
annotation that is included in the same electronic mail message as well as the 
corresponding media content to be rendered. 

(7) Grouping of Claims 

All of the claims do not stand or fall together. The claims are grouped as 
follows: 

Group I: Claims 1 and 3-5. 
Group II: Claims 2, 37-39, and 41-42. 
Group III: Claims 17-25, and 27-30. 
Group IV: Claims 31 and 32. 
Group V: Claims 33 and 35-36. 
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Group VI: Claims 13-16, and 40. 
(8) Argument 

1. Group I: Claims 1 and 3-5 are not taught or suggested by Hou 
in view of Sidana because the combination of Hou and Sidana does not 
disclose or suggest an annotation server to analyze electronic mail messages 
that are received not in reply to a previous electronic mail message that 
included annotation data, to locate data in the electronic mail messages, and 
to generate new annotations that include the located data but no new 
annotation data. 

In the Final Office Action dated June 3, 2003, claims 1 and 3-5 were 
rejected under 35 U.S.C. § 103(a) as being unpatentable over Hou (U.S. Patent 
No. 5,838,313) in view of Sidana (U.S. Patent No. 6,081,829). 

Hou discloses a multimedia-based reporting system that allows drawing, 
text, and audio recording annotations to be added to the report (see, col. 2, lines 
40-61, and col. 6, lines 52-66). The reporting system of Hou includes an 
annotation input interpreter that provides an interface between the user annotation 
input and three types of annotation handlers that can add annotations to a "canvas" 
of the report: a static annotation handler, a dynamic annotation handler, and an 
annotation on annotation handler (see, Fig. 9, and col. 6, lines 25-29). 

Sidana is directed to general purpose web annotations without modifying 
the browser. In Sidana, a "redirector" is situated between a client browser and a 
server including a web document (see, Fig. 1). The redirector stores the user's 
annotations to the web document, so the next time the user's client browser 
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requests the web document, the request is routed through the redirector which adds 
the user's previous annotations to create an augmented web document that is 
displayed to the user. See, col. 4, lines 57-67, and col. 1 , line 55 - col. 2, line 32. 
In contrast, claim 1 recites in part: 

an annotation server, coupled to the client computer, to, 

maintain an annotation database having a plurality of 

annotations corresponding to the multimedia content, 

provide the plurality of annotations to the client computer for 

playback, 

send electronic mail messages including annotations to 
recipients identified by the client computer, 

analyze electronic mail messages that are received not in 
reply to a previous electronic mail message that included annotation 
data, to locate data in the electronic mail messages, 

generate new annotations that include the located data but no 
new annotation data, and 

add the new annotations to the annotation database. 

Hou is cited as teaching "generate new annotation that include display 
located content data in the electronic mail message (e.g., based on existent 
annotation data displayed col 6/lines 21-40); but no new annotation data" and "add 
the new annotations saved in annotation database (col 4/lines 21-25, save report 
col 2/lines 50-51, report includes multimedia, col 1 /lines 39-44); but no new 
annotation data (see Fig. 6, col 5/lines 15-25)" (see, June 3, 2003 Final Office 
Action at H 3, p. 3). 

Appellant respectfully submits, however, that Hou does not disclose or 
suggest to analyze electronic mail messages, generate new annotations, and add 
the new annotation to an annotation database as recited in claim 1 . 

Hou discloses an annotation on annotation handler that allows a user to 
select a desired segment of an annotation and make a new annotation which is a 
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reply to this selected segment (see, col. 9, lines 14-24). However, this annotation 
on annotation handler of Hou allows a new annotation to be made that is a reply to 
a selected segment. Thus, the new annotations made using the annotation on 
annotation handler of Hou do not satisfy the "analyze electronic mail messages 
that are received not in reply to a previous electronic mail message that included 
annotation data, to locate data in the electronic mail messages" language of claim 
1 because the new annotations made using the annotation on annotation handler of 
Hou are a reply. 

Furthermore, the new annotations made using the annotation on annotation 
handler of Hou do not satisfy the "generate new annotations that include the 
located data but no new annotation data" language of claim 1 because the new 
annotations made using the annotation on annotation handler of Hou do include 
new annotation data. 

Hou also discloses that, as illustrated in Figure 9 of Hou, annotations are 
created in Hou by way of user annotation input 36 that is provided to an 
annotation input interpreter, which forwards the input to either static annotation 
handler 37, annotation on annotation handler 38, or dynamic annotation handler 
39. These three handlers 37, 38, and 39 allow for the addition of different 
annotation inputs (e.g., drawings, text, or voice) to the report 32 (see also, col. 6, 
lines 26-37). After all annotations have been generated and added to report 32 by 
the interpreter 24 and handlers 37, 38, and 39, mail report handler can generate an 
e-mail message for delivery (see also, col. 6, lines 14-16, and col. 4, lines 25-31). 

Thus, Hou discloses generating annotations and adding them to a report, 
and then generating an email message including the report . In other words, Hou 
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discloses generating the annotations and then generating the email message, 
whereas in claim 1 the electronic mail messages are analyzed and the new 
annotations generated to include the located data in the electronic mail messages. 
As the annotations in Hou are generated prior to generating the email message of 
Hou 5 Appellant respectfully submits that Hou cannot disclose or suggest 
generating new annotations that include located data from analyzing electronic 
mail messages (the email message is generated after the annotation in Hou, so 
there is no message to analyze in Hou). 

Sidana is not cited as curing these deficiencies of Hou, and Appellant 
respectfully submits that Sidana does not cure these deficiencies of Hou. Thus, 
given that neither Hou nor Sidana discloses or suggests an annotation server to 
analyze electronic mail messages that are received not in reply to a previous 
electronic mail message that included annotation data, to locate data in the 
electronic mail messages, and to generate new annotations that include the located 
data but no new annotation data, Appellant respectfully submits that the 
combination of Hou and Sidana does not disclose or suggest an annotation server . 
to analyze electronic mail messages that are received not in reply to a previous 
electronic mail message that included annotation data, to locate data in the 
electronic mail messages, and to generate new annotations that include the located 
data but no new annotation data as recited in claim 1 . 

In the June 3, 2003 Final Office Action, it was asserted that "In response to 
applicants argument that the references fail to show certain features of applicant's 
invention, it is noted that the features upon which applicant relies (i.e., generating 
an electronic mail message not in reply to a previous electronic mail message, nor 
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generating a new annotation which is a not reply to a selected segment) are not 
recited in the rejected claim(s)." (see, June 3, 2003 Final Office Action at ^| 6, p. 

10) . Applicant respectfully disagrees with this assertion, and respectfully points to 
the preceding discussion which specifically points out elements of claim 1 that are 
not disclosed or suggested by the cited references. 

In the June 3, 2003 Final Office Action, in response to Appellant's 
arguments regarding claim 1 and Hou, it was asserted that "Applicant's 
interpretation of the prior art is noted, however Hou teaches locating data in a 
received electronic mail message (col 4/lines 10-14) and saving located data in a 
database (col 4/lines 21-25);" (see, June 3, 2003 Final Office Action at K 7, pp. 10- 

11) . Hou at col. 4, lines 10-14 discusses deciding whether a report is from a 
mailbox, and if so a message list is displayed, the user selects a message and the 
report is loaded from the message. 

However, Appellant respectfully submits that there is nothing in this cited 
portion of Hou that discloses or suggests the "analyze electronic mail messages 
that are received not in reply to a previous electronic mail message that included 
annotation data, to locate data in the electronic mail messages" language and the 
"generate new annotations that include the located data but no new annotation 
data" language of claim 1 . As discussed above, Hou discloses generating the 
annotations and then generating the email message, whereas in claim 1 the 
electronic mail messages are analyzed and the new annotations generated to 
include the located data in the electronic mail messages. As such, the disclosure at 
col. 4, lines 10-14 of Hou does not disclose or suggest the analyze and generate 
language of claim 1 . 



12 



Application No. 09/396/701 



For at least these reasons, Appellant respectfully submits that claim 1 is 
allowable over Hou in view of Sidana. 

Accordingly, Appellant respectfully submits that claims 1 and 3-5 are 
allowable over the cited references and that the rejection should be withdrawn. 

2. Group II: Claims 2, 37-39, and 41-42 are not taught or 
suggested by Hou in view of Sidana in further view of Birrell because the 
combination of Hou, Sidana, and Birrell does not disclose or suggest an 
annotation server to analyze electronic mail messages that are received not in 
reply to a previous electronic mail message that included annotation data, to 
locate data in the electronic mail messages, and to generate new annotations 
that include the located data but no new annotation data. 

Birrell is directed to replacing a large bit component of an electronic mail 
message with a hot-link (see, title). In Birrell, a mail message stored in a mail 
service system includes a primary component and a secondary component (see, 
col. 2, lines 1-5). When the mail message is requested from a client computer 
system, the secondary component is replaced with a hot-link (see, col. 2, lines 6- 
10). The secondary component is sent to the client computer when a user of the 
client computer clicks on the hot-link (see, col. 2, lines 1 5-17). 

In the Final Office Action dated June 3, 2003, claims 2, 37-39, and 41-42 
were rejected under 35 U.S.C. § 103(a) as being unpatentable over Hou (U.S. 
Patent No. 5,838,313) in view of Sidana (U.S. Patent No. 6,081,829) and in further 
view of Birrell (U.S. Patent No. 6,009,462). 
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With respect to claims 2 and 42, claims 2 and 42 depend from claim 1. 
Birrell is not cited as curing the deficiencies of Hou and Sidana as discussed above 
with respect to claim 1, and Appellant respectfully submits that Birrell does not 
cure these deficiencies of Hou and Sidana. Thus, Appellant respectfully submits 
that claims 2 and 42 are allowable over Hou in view of Sidana and in further view 
of Birrell for at least the reasons discussed above with reference to claim 1. 

With respect to claims 37-39, Appellant respectfully submits that, 
analogous to the discussion above regarding claim 1, Hou does not disclose or 
suggest receiving an electronic mail message that is not in reply to a previous 
electronic mail message; locating data in the received electronic mail message; 
generating a new annotation from the located data in the electronic mail message, 
wherein the new annotation includes the located data but no new annotation data; 
and adding the new annotation to an annotation store as recited in claim 37. 
Sidana and Birrell are not cited as curing these deficiencies of Hou, and Appellant 
respectfully submits that Sidana and Birrell do not cure these deficiencies of Hou. 
For at least these reasons, Appellant respectfully submits that claim 37 is 
allowable over Hou in view of Sidana and in further view of Birrell. 

With respect to claim 41, Appellant respectfully submits that, analogous to 
the discussion above regarding claim 1, Hou does not disclose or suggest receiving 
an electronic mail message including data corresponding to media content that is 
not included in the electronic mail message; locating the data in the electronic mail 
message; generating, after locating the data, a new annotation corresponding to the 
media content, wherein the new annotation includes a content field including the 
located data but no new annotation data as recited in claim 41. Sidana and Birrell 
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are not cited as curing these deficiencies of Hou 5 and Appellant respectfully 
submits that Sidana and Birrell do not cure these deficiencies of Hou. For at least 
these reasons, Appellant respectfully submits that claim 41 is allowable over Hou 
in view of Sidana and in further view of Birrell. 

Accordingly, Appellant respectfully submits that claims 2, 37-39, and 41- 
42 are allowable over the cited references and that the rejection should be 
withdrawn. 

3. Group III: Claims 17-25 and 27-30 are not taught or suggested 
by Hou in view of Sidana in further view of Birrell because the combination 
of Hou, Sidana, and Birrell does not disclose or suggest media content, being 
accessed for rendering to a user, beginning with the one of a plurality of 
temporal segments of the media content that tdrresponds to a new 
annotation. 

In the Final Office Action dated June 3, 2003, claims 17-25 and 27-30 were 
rejected under 35 U.S.C. § 103(a) as being unpatentable over Hou (U.S. Patent 
No. 5,838,313) in view of Sidana (U.S. Patent No. 6,081,829) and in further view 
of Birrell (U.S. Patent No. 6,009,462). 

With respect to claim 17, claim 17 is directed to a method comprising; 

receiving an electronic mail notification of a new annotation 
corresponding to media content, the new annotation corresponding to one 
of a plurality of temporal segments of the media content that is after the 
beginning of the media content; 

presenting the electronic mail notification to a user; 

receiving a user selection of an identifier, within the electronic mail 
notification, of the media content; and 
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accessing, in response to the user selection, a media server to stream, 
for rendering to the user, the media content beginning with the one of the 
plurality of temporal segments that corresponds to the new annotation. 

In the June 3, 2003 Final Office Action it was asserted that the receiving 
and accessing were taught by Hou and Birrell as follows (see, June 3, 2003 Final 
Office Action at^|4, p. 6): 

receiving user selection of an identifier within the electronic 
mail message ("notification"), accessing in response to the user 
selection a media server to download or retrieve on demand (i.e. 
stream) rendering to the user (Birrell: user selection, col 12/lines 22- 
27, col 13/lines 1-7, access media server: col 4/lines 27-30, 60-61, 
server for receiving messages, col 1 4/lines 14-18, receiving mail 
messages, col 3/lines 48-49), the media content beginning with one 
of a plurality of temporal segment markers that correspond to the 
media content (Hou: col 8/lines 43-48, 61-63, col 9/lines 8-13). 

Appellant disagrees with this assertion and respectfully submits that Hou in view 
of Sidana and further in view of Birrell does not disclose the method of claim 17. 

As discussed above, Birrell is directed to replacing a large bit component of 
an electronic mail message with a hot-link (see, title), and then sending that 
component to a client computer when a user of the client computer clicks on the 
hot-link. Nothing in the discussion of this link in Birrell discloses or suggests that 
when a user clicks on the hot-link the large bit component starts being rendered at 
any particular location of the component, much less of media content, being 
accessed for rendering to a user, beginning with the one of a plurality of temporal 
segments of the media content that corresponds to a new annotation as recited in 
claim 17. 

With respect to Hou, as discussed above Hou discloses annotations that can 
be generated and added to a report. However, nowhere in Hou is there any 
disclosure or suggestion of media content, being accessed for rendering to a user, 
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beginning with the one of a plurality of temporal segments of the media content 
that corresponds to a new annotation as recited in claim 17. Appellant respectfully 
submits that merely adding an annotation to a report does not disclose or suggest 
media content, being accessed for rendering to a user, beginning with the one of a 
plurality of temporal segments of the media content that corresponds to a new 
annotation as recited in claim 17. 

Sidana is not cited as curing these deficiencies of Hou and Birrell, and 
Appellant respectfully submits that Sidana does not cure these deficiencies of Hou 
and Birrell. Thus, given that none of Hou, Birrell, and Sidana discloses or 
suggests media content, being accessed for rendering to a user, beginning with the 
one of a plurality of temporal segments of the media content that corresponds to a 
new annotation, Appellant respectfully submits that the combination of Hou, 
Birrell, and Sidana does not disclose or suggest media content, being accessed for 
rendering to a user, beginning with the one of a plurality of temporal segments of 
the media content that corresponds to a new annotation as recited in claim 17. 

For at least these reasons, Appellant respectfully submits that claim 17 is 
allowable over Hou in view of Sidana and in further view of Birrell. 

Accordingly, Appellant respectfully submits that claims 17-25 and 27-30 
are allowable over the cited references and that the rejection should be withdrawn. 

4. Group IV: Claims 31 and 32 are not taught or suggested by Hou 
in view of Sidana in further view of Birrell because the combination of Hou, 
Sidana, and Birrell does not disclose or suggest displaying a default recipient, 
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corresponding to the identified one or more annotation sets, that is to receive 
an electronic mail notification of the new annotation. 

In the Final Office Action dated June 3, 2003, claims 31 and 32 were 
rejected under 35 U.S.C. § 103(a) as being unpatentable over Hou (U.S. Patent 
No. 5,838,313) in view of Sidana (U.S. Patent No. 6,081,829) and in further view 
of Birrell (U.S. Patent No. 6,009,462). 

With respect to claim 3 1 , claim 3 1 is directed to a method comprising: 

receiving an electronic mail notification of an annotation 
corresponding to media content; 

replying to the electronic mail notification to generate a new 
annotation corresponding to the media content; 

including, in the reply, an identifier of one or more annotation sets of 
a plurality of annotation sets that the new annotation is associated with, 
wherein the annotation sets correspond to categories for annotations; and 

displaying a default recipient, corresponding to the identified one or 
more annotation sets, that is to receive an electronic mail notification of the 
new annotation. 

In the June 3, 2003 Final Office Action, it was asserted that the prior art 
teaches "corresponding to identified annotation set (Hou: annotation set: col 
7/lines 1-6, annotation segments, col 8/lines 43-48, col 9/lines 8- 13)" (see, June 3, 
2003 Final Office Action at ^ 9, p. 12). It appears from this assertion that the 
event data of Hou is being relied on as teaching the annotation sets as recited in 
claim 3 1 . Appellant respectfully submits that Hou does not disclose or suggest the 
annotation sets as recited in claim 31. 

In claim 3 1 , a default recipient that is to receive an electronic mail 
notification of a new annotation corresponds to an identified one or more 
annotation sets. Appellant respectfully submits that no such default recipient 
exists in Hou. Using the language of Hou, if the event data of Hou were to be the 
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annotation sets, then Hou would need to disclose a default recipient that is to 
receive an electronic mail notification of a new annotation that corresponds to an 
identified one or more event data. Appellant respectfully submits that there is no 
disclosure or suggestion of the event data of Hou corresponding to a default 
recipient, so the event data cannot disclose or suggest the annotation sets of claim 
31, much less of displaying a default recipient, corresponding to the identified one 
or more annotation sets, that is to receive an electronic mail notification of the new 
annotation as recited in claim 3 1 . 

Sidana and Birrell are not cited as curing these deficiencies of Hou, and 
Appellant respectfully submits that Sidana and Birrell do not cure these 
deficiencies of Hou. Thus, given that none of Hou, Birrell, and Sidana discloses 
or suggests displaying a default recipient, corresponding to the identified one or 
more annotation sets, that is to receive an electronic mail notification of the new 
annotation, Appellant respectfully submits that the combination of Hou, Birrell, 
and Sidana does not disclose or suggest displaying a default recipient, 
corresponding to the identified one or more annotation sets, that is to receive an 
electronic mail notification of the new annotation as recited in claim 3 1 . 

For at least these reasons, Appellant respectfully submits that claim 3 1 is 
allowable over Hou in view of Sidana and in further view of Birrell. 

Accordingly, Appellant respectfully submits that claims 31 and 32 are 
allowable over the cited references and that the rejection should be withdrawn. 

5. Group V: Claims 33 and 35-36 are not taught or suggested by 
Hou in view of Sidana in further view of Birrell because the combination of 
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Hou, Sidana, and Birrell does not disclose or suggest creating, after the 
plurality of electronic mail messages have been included in the electronic mail 
thread, a plurality of annotations from the plurality of electronic mail 
messages in the electronic mail thread. 

In the Final Office Action dated June 3, 2003, claims 33 and 35-36 were 
rejected under 35 U.S.C. § 103(a) as being unpatentable over Hou (U.S. Patent 
No. 5,838,313) in view of Sidana (U.S. Patent No. 6,081,829) and in further view 
of Birrell (U.S. Patent No. 6,009,462). 

With respect to claim 33, claim 33 is directed to a method comprising: 

receiving an electronic mail thread including a plurality of electronic 
mail messages; 

creating, after the plurality of electronic mail messages have been 
included in the electronic mail thread, a plurality of annotations from the 
plurality of electronic mail messages in the electronic mail thread; and 

adding the plurality of annotations to an annotation database. 

Appellant respectfully submits that Hou in view of Sidana and in further 
view of Birrell does not disclose or suggest creating, after the plurality of 
electronic mail messages have been included in the electronic mail thread, a 
plurality of annotations from the plurality of electronic mail messages in the 
electronic mail thread as recited in claim 33. 

In the June 3, 2003 Final Office Action, Birrell is cited as teaching 
"creating a plurality of annotations (Birell: col 8/lines 32-37, annotation including 
media content col 12/lines 28-31, 52-53, add annotation, col 14/line 14) on 
multiple electronic mail message(s) received or created electronic mail messages, 
receiving electronic mail messages including electronic mail thread which 
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includes a plurality of electronic mail messages (Birell: col 1 1 /lines 22-30, col 
13/lines 25-32)" (see, June 3, 2003 Final Office Action at ^4, p. 7). 

It appears that the labels of Birrell (see, col. 8, lines 32-37) are being relied 
on as teaching the annotations of claim 33. As discussed in Birrell, a label is 
similar to affixing a note to a printed document, and the labels can be used to 
replace the folder mechanisms used by many prior art mail systems (see, col. 8, 
lines 32-37). Additionally, a single mail message can have multiple labels (see, 
col. 8, lines 32-37). Furthermore, labels can be added and removed by the system 
or by users (see, col. 8, lines 45-46). A menu bar that includes an "Add" button 
used to add a selected label to a message is also disclosed (see, col. 14, lines 11- 
14). 

Using the language of claim 33, if the labels of Birrell were the annotations 
of claim 33, then Birrell would have to disclose creating, after the plurality of 
electronic mail messages have been included in the electronic mail thread, a 
plurality of labels from the plurality of electronic mail messages in the electronic 
mail thread. Applicant respectfully submits that nowhere in Birrell is there any 
discussion or suggestion of creating a plurality of labels from the plurality of 
electronic mail messages in an electronic mail thread. Although Birrell discloses 
that labels can be attached to messages (see, col. 9, lines 25-30), and that labels 
can be added and removed, the mere attachment or addition or removal of labels 
does not disclose or suggest creating a plurality of labels from the plurality of 
electronic mail messages in the electronic mail thread. Thus, Appellant 
respectfully submits that the labels of Birrell cannot disclose or suggest the 
annotations of claim 33, much less of creating, after the plurality of electronic mail 
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messages have been included in the electronic mail thread, a plurality of 
annotations from the plurality of electronic mail messages in the electronic mail 
thread as recited in claim 33. 

Sidana and Hou are not cited as curing these deficiencies of Birrell, and 
Appellant respectfully submits that Sidana and Hou do not cure these deficiencies 
of Birrell. Thus, given that none of Hou, Birrell, and Sidana discloses or suggests 
creating, after the plurality of electronic mail messages have been included in the 
electronic mail thread, a plurality of annotations from the plurality of electronic 
mail messages in the electronic mail thread, Appellant respectfully submits that 
the combination of Hou, Birrell, and Sidana does not disclose or suggest creating, 
after the plurality of electronic mail messages have been included in the electronic 
mail thread, a plurality of annotations from the plurality of electronic mail 
messages in the electronic mail thread as recited in claim 33. 

For at least these reasons, Appellant respectfully submits that claim 33 is 
allowable over Hou in view of Sidana and in further view of Birrell. 

Accordingly, Appellant respectfully submits that claims 33 and 35-36 are 
allowable over the cited references and that the rejection should be withdrawn. 

6. Group VI: Claims 13-16, and 40 are not taught or suggested by 
Hou in view of Sidana in view of Birrell in further view of Russell because the 
combination of Hou, Sidana, Birrell, and Russell does not disclose or suggest 
including an installation option in an electronic mail message that identifies 
an installation program that can be used to install one or more modules 
allowing the content of a new annotation that is included in the same 
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electronic mail message as well as the corresponding media content to be 
rendered. 

In the Final Office Action dated June 3, 2003, claims 13-16 were rejected 
under 35 U.S.C. § 103(a) as being unpatentable over Hou (U.S. Patent No. 
5,838,313) in view of Sidana (U.S. Patent No. 6,081,829) in view of Birrell (U.S. 
Patent No. 6,009,462) and in further view of Russell (U.S. Patent No. 5,526,407). 
Claim 40 was rejected under 35 U.S.C. § 103(a) as being unpatentable over Hou in 
view of Sidana and in further view of Birrell. 

With respect to claim 13, claim 13 recites in part: 

generating an electronic mail message including, 

an installation option that identifies an installation program that can 
be used to install one or more modules allowing the content and 
corresponding media content to be rendered; 

Appellant respectfully submits that the cited references do not disclose or 
suggest generating an electronic mail message that includes an installation option 
as recited in claim 13. 

In the June 3, 2003 Final Office Action, at ^ 1 1, pp. 12-13, it was asserted 
that Russell and Birrell teach "generating an electronic mail message that includes 
an installation option" as follows: 

Russell teaches a user selecting (col 19/lines 44-49, 3-15) the 
executing of an program (col 17/lines 21-25) the installs modules 
upon demand ("installation option") that identifies an installation 
program that can be used to dynamically install module(s) (DDL) 
(col 16/lines 63-col 17/line 6) allowing the content and 
corresponding media content to be animated display of rendered 
(119) (col 17/lines 54-56, col 18/lines 25-29). 

Birrell teaches an composing electronic mail message(s) (col 
4/lines 32-35, 48-49) having annotations (col 8/lines 36-37, 46-47) 
annotations include media content and/or embedded HTML files 
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(col 12/lines 52-67, col 2/lines 1-15, col 9/lines 31-35); installation 
options such as software program files (e.g. applets, plug-ins) 
downloadable over the network, implemented using HTML, 
JavaScript's or Java applets files (col 2/lines 66-col 3/line 8), 
downloaded program for rendering multimedia content at the client 
(col 12/lines 1-11), some content file formats included in the 
electronic mail message' can directly be displayed (rendered) or by 
configuring the browser with a "helper" applet to "display" specific 
formats (col 12/lines 28-44). 

Appellant respectfully submits that nowhere in Russell or Birrell is there 
any discussion of including an installation option in an electronic mail message as 
recited in claim 13. Merely having an application installed on a personal computer 
does not disclose or suggest including an installation option in an electronic mail 
message that identifies an installation program that can be used to install one or 
more modules allowing the content of a new annotation that is included in the 
same electronic mail message as well as the corresponding media content to be 
rendered as recited in claim 13. Nor does having software program files 
downloadable over a network, or electronic mail messages, disclose this aspect of 
claim 13. 

Because including an installation option in an electronic mail message as 
recited in claim 13 is not disclosed or suggested in Russell or Birrell, Appellant 
respectfully submits that neither Russell nor Birrell can disclose or suggest the 
installation option of claim 13. 

Sidana and Hou are not cited as curing these deficiencies of Birrell and 
Russell, and Appellant respectfully submits that Sidana and Hou do not cure these 
deficiencies of Birrell and Russell. Thus, given that none of Hou, Sidana, Birrell, 
and Russell discloses or suggests including an installation option in an electronic 
mail message that identifies an installation program that can be used to install one 
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or more modules allowing the content of a new annotation that is included in the 
same electronic mail message as well as the corresponding media content to be 
rendered, Appellant respectfully submits that the combination of Hou, Sidana, 
Birrell, and Russell does not disclose or suggest including an installation option in 
an electronic mail message that identifies an installation program that can be used 
to install one or more modules allowing the content of a new annotation that is 
included in the same electronic mail message as well as the corresponding media 
content to be rendered as recited in claim 13. 

For at least these reasons, Appellant respectfully submits that claim 13 is 
allowable over Hou in view of Sidana and in view of Birrell and further in view of 
Russell. 

Accordingly, Appellant respectfully submits that claims 13, 14-16, and 40 
are allowable over the cited references and that the rejection should be withdrawn. 

Conclusion 

The Office's basis and supporting rationale for the § 103(a) rejections is not 
supported by the teaching of the cited references. Applicant respectfully requests 
that the rejections be overturned and that pending claims 1-5, 13-25, 27-33, and 
35-42 be allowed to issue. 




Respectfully Submitted, 



jK\\an T. Sponseller 
Lee & Hayes, PLLC 
Reg. No. 38,318 
(509) 324-9256 ext. 215 
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(9) Appendix of Appealed Claims 

1 . A system comprising: 
a client computer to playback multimedia content and annotations 

corresponding to different temporal portions of the multimedia content; and 
an annotation server, coupled to the client computer, to, 

maintain an annotation database having a plurality of annotations 
corresponding to the multimedia content, 

provide the plurality of annotations to the client computer for 
playback, 

send electronic mail messages including annotations to recipients 
identified by the client computer, 

analyze electronic mail messages that are received not in reply to a 
previous electronic mail message that included annotation data, to locate 
data in the electronic mail messages, 

generate new annotations that include the located data but no new 
annotation data, and 

add the new annotations to the annotation database. 

2. A system as recited in claim 1, further comprising a media server to 
manage streaming the multimedia content to the client computer, wherein 
streaming the multimedia content comprises the multimedia content being 
communicated to the client computer on an as-needed basis rather than being pre- 
delivered in its entirety prior to beginning playback of the multimedia content. 
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3. A system as recited in claim 1, further comprising a network 
coupling the client computer to the annotation server. 

4. A system as recited in claim 1, wherein the client computer is further 

to: 

present an electronic mail message including a multimedia content 
identifier to a user, 

receive a user selection of the multimedia content identifier, 
access a media server to obtain the multimedia content, and 
playback the multimedia content to the user. 

5. A system as recited in claim 4, wherein the client computer is further 
to transmit, to the media server, an identifier of a temporal segment of the 
multimedia content that is after the beginning of the multimedia content, and 
wherein the media server is to stream to the client computer the multimedia 
content beginning with the identified temporal segment. 
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13. A method comprising: 

receiving data for a new annotation corresponding to a temporal range of 
media content; 

generating an electronic mail message including, 

the content of the new annotation, 

an identifier of the media content, and 

an installation option that identifies an installation program that can 
be used to install one or more modules allowing the content and 
corresponding media content to be rendered; and 

forwarding the electronic mail message to a recipient identified by the data. 

14. A method as recited in claim 13, wherein the generating further 
comprises including, in the electronic mail message, an identifier of the new 
annotation. 

15. A method as recited in claim 13, wherein the generating further 
comprises including, in the electronic mail message, an identifier of the temporal 
range of media content. 

16. One or more computer-readable memories containing a computer 
program that is executable by a processor to perform the method recited in claim 
13. 
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17. A method comprising: 

receiving an electronic mail notification of a new annotation corresponding 
to media content, the new annotation corresponding to one of a plurality of 
temporal segments of the media content that is after the beginning of the media 
content; 

presenting the electronic mail notification to a user; 

receiving a user selection of an identifier, within the electronic mail 
notification, of the media content; and 

accessing, in response to the user selection, a media server to stream, for 
rendering to the user, the media content beginning with the one of the plurality of 
temporal segments that corresponds to the new annotation. 

18. A method as recited in claim 17, wherein the presenting comprises 
displaying annotation content for the new annotation to the user. 
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19. A method as recited in claim 17, wherein the presenting comprises: 
displaying annotation content for the new annotation to the user; and 
displaying a plurality of identifiers to the user, the plurality of identifiers 
including: 

a content identifier that identifies the media content; 

a range identifier that identifies a temporal range of the segment 
corresponding to the new annotation; 

an annotation identifier that identifies the new annotation; and 

an annotation set identifier that identifies one or more of a plurality 
of annotation sets, corresponding to categories for annotations, that the new 
annotation is part of. 

• 20. A method as recited in claim 19, wherein the content identifier 
comprises a uniform resource locator (URL). 

21. A method as recited in claim 17, wherein the receiving a user 
selection comprises receiving a user selection of a uniform resource locator (URL) 
of the media content. 

22. A method as recited in claim 17, wherein the receiving a user input 
comprises receiving a user selection of an identifier of the annotation. 
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23. One or more computer-readable media having stored thereon a 
computer program that, when executed by one or more processors, causes the one 
or more processors to perform functions including: 

receiving an electronic mail notification of a new annotation corresponding 
to media content, wherein the electronic mail notification includes a user- 
selectable identifier of the media content; 

displaying the electronic mail notification; 

receiving a user selection of the identifier; and 

automatically accessing, in response to the user selection of the identifier of 
the media content, a media server indicated by the identifier to begin, at a point in 
the media content after the beginning of the media content and corresponding to 
the new annotation, streaming the media content for presentation to the user. 

24. One or more computer-readable media as recited in claim 23, 
wherein the user-selectable identifier comprises a uniform resource locator (URL) 
that identifies a server and a location at the server where the media content is 
located. 
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25. A system comprising: 

an interface module to receive data regarding a new annotation 
corresponding to media content; 

a module to generate an electronic mail message regarding the new 
annotation, the electronic mail message including, 
the new annotation content, 

a user-selectable link to the media content, wherein the link includes 
an identifier of the media content to which the new annotation corresponds, 
and an identifier of a temporal range of the media content that the new 
annotation content is associated with, wherein the temporal range 
corresponds to a location of the media content that is after the beginning of 
the media content and at which rendering of the media content should begin 
in response to selection of the link in the electronic mail message. 

27. A system as recited in claim 25, wherein the electronic mail message 
further includes a unique identifier of the new annotation. 

28. A system as recited in claim 25, wherein the electronic mail message 
further includes an identifier of one or more of a plurality of annotation sets, 
corresponding to categories for annotations, that the new annotation is associated 
with. 

29. A system as recited in claim 25, wherein the system comprises an 
annotation server computer. 
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30. A system as recited in claim 25, wherein the system comprises a 
client computer and wherein the interface module comprises a user interface. 

31. A method comprising: 

receiving an electronic mail notification of an annotation corresponding to 
media content; 

replying to the electronic mail notification to generate a new annotation 
corresponding to the media content; 

including, in the reply, an identifier of one or more annotation sets of a 
plurality of annotation sets that the new annotation is associated with, wherein the 
annotation sets correspond to categories for annotations; and 

displaying a default recipient, corresponding to the identified one or more 
annotation sets, that is to receive an electronic mail notification of the new 
annotation. 

32. A method as recited in claim 31, wherein the replying includes: 
obtaining, from the electronic mail notification, an identifier of the 

annotation; and 

including the identifier of the annotation in the reply. 
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33. A method comprising : 

receiving an electronic mail thread including a plurality of electronic mail 

messages; 

creating, after the plurality of electronic mail messages have been included 
in the electronic mail thread, a plurality of annotations from the plurality of 
electronic mail messages in the electronic mail thread; and 

adding the plurality of annotations to an annotation database. 

35. A method as recited in claim 33, wherein: 

the creating comprises generating, for each of the plurality of electronic 
mail messages, an annotation; and 

the adding comprises adding each of the generated annotations to the 
annotation database. 

36. A method as recited in claim 33, further comprising locating, in the 
electronic mail thread, an identifier of media content that the annotation 
corresponds to. 
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37. One or more computer-readable media having stored thereon a 
computer program that, when executed by one or more processors, causes the one 
or more processors to perform functions including: 

receiving an electronic mail message that is not in reply to a previous 
electronic mail message; 

locating data in the received electronic mail message; 

generating a new annotation from the located data in the electronic mail 
message, wherein the new annotation includes the located data but no new 
annotation data; and 

adding the new annotation to an annotation store. 

38. One or more computer-readable media as recited in claim 37, 
wherein the computer pfogram further causes the one or more processors to 
perform functions including: 

creating a new annotation based on the extracted annotation content and the 
annotation identification information; and 

adding the new annotation to an annotation database. 
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39. One or more computer-readable media as recited in claim 37, 
wherein the located data comprises: 

an identifier of media content to which the annotation content corresponds 
and of a temporal segment, corresponding to the annotation content, of the media 
content; and 

an identifier of an annotation set that a new annotation including the 
extracted annotation content is to be part of, wherein the annotation set is one of a 
plurality of annotation sets corresponding to categories for annotations. 

40. A method as recited in claim 13, wherein the installation option 
comprises a user-selectable link that identifies the installation program. 
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41. A method comprising: 

receiving an electronic mail message including data corresponding to media 
content that is not included in the electronic mail message; 
locating the data in the electronic mail message; 

generating, after locating the data, a new annotation corresponding to the 
media content, wherein the new annotation includes, 

an author field including data identifying a sender of the electronic 
mail message as an author of the new annotation, 

a creation time field including data identifying the time at which the 
new annotation is generated, 

a title field including data identifying a title of the new annotation, 

a content field including the located data but no new annotation data, 

and 

a media content identifier field including data identifying the media 
content to which the new annotation corresponds. 
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42. A system as recited in claim 1, wherein each generated new 
annotation includes: 

an author field including data identifying an author of the new annotation, 

a creation time field including data identifying the time at which the new 
annotation is generated, 

a title field including data identifying a title of the new annotation, 

a content field including the located data, and 

a media content identifier field including data identifying the multimedia 
content to which the new annotation corresponds. 
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